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(54) Image processing apparatus, image processing method and storage medium 



(57) An image processing apparatus including: an 
image acquiring element for acquiring a target image; 
an identification information recognizing element for 
recognizing identification information corresponding to 
a specific image pattern from the target image acquired 



by the image acquiring element; and an activating ele- 
ment for activating selectively from among a plurality of 
previously stored processes a specific process corre- 
sponding to the identification information recognized by 
the identification information recognizing element so as 
to start execution of the specific process. 
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Description 

[0001] The present invention relates to an image 
processing apparatus, an image processing method, 
and a storage medium for allowing an image processing 
program to be executed by a computer. 
[0002] Recent years have witnessed widespread use 
of personal computer operating systems such as Win- 
dows 95 and Windows 98 (registered trademarks), both 
from Microsoft Corporation in the United States. Such 
operating systems offer on a display screen a task bar 
with indications visible to users such as buttons repre- 
senting applications each having an active window. A 
technique dealing with the use of the task bar is dis- 
closed in detail by Japanese Patent Laid-open No. Hei 
8-255066. 

[0003] Below is an excerpt from the cited patent ap- 
plication. The task bar is a visible user interface element 
that informs a user which tasks are currently active. The 
task bar also serves as a point for displaying currently 
active task windows. The task bar includes a start menu 
button including menu items that allow the user to acti- 
vate desired programs, open documents and control 
system settings. 

[0004] In a typical computer system including a cen- 
tral processing unit (CPU) and such peripherals as a 
mouse, a keyboard and a video display, the start menu 
button on the task bar is operated illustratively as fol- 
lows: the start menu button is operated by the user to 
open the start menu serving as a centralized location for 
gaining access to programs, documents, system con- 
trols and help information. Initially, the user points a 
mouse cursor to the start menu button and clicks on a 
left mouse button. The actions cause the start menu to 
appear. 

[0005] The start menu typically includes such menu 
items as programs, search, setting and help. The "pro- 
grams'' menu item permits access to a program menu 
hierarchically displayed from the start menu become ac- 
cessible. The program menu displays a plurality of ap- 
plication programs and a group of programs that may 
be selected by the user. 

[0006] The user is required repeatedly to perform 
complicated actions when selecting and activating a de- 
sired application program from the task bar including the 
start menu button. Specifically, the user first points the 
mouse cursor to the start menu button on the task bar 
and clicks on the left mouse button to display the start 
menu. Next, the user points the mouse cursor to the 
"programs" menu item in the start menu and clicks on 
the left mouse button to display the program menu. The 
user then points the mouse cursor to an indication of the 
desired application program in the program menu, and 
clicks on the left mouse button. Another series of point 
and click actions is needed if there is an application pro- 
gram group. Finally the CPU activates the selected ap- 
plication program. 

[0007] The user is required conventionally to repeat 



such complicated act ions before starting the desired ap- 
plication program. If the user is already handling some 
other chores with one hand, operating the above point- 
ing device with the other hand often turns out to be a 
5 difficult task from a user interface point of view. The 
same difficulty is encountered when other menu items 
such as "help" are clicked on from the start menu pro- 
gram for desired processing. 

[0008] Meanwhile, the so-called bar code system is 
io used extensively in various industrial fields. The system 
involves encoding in bars alphanumeric characters rep- 
resenting identification numbers and other information 
about goods and articles, and attaching such codes to 
the goods and products so that their identification infor- 
ms mation may laterbe scanned and retrieved by an optical 
recognition apparatus called a bar code scanner. 
[0009] At cash registers or like equipment installed in 
retail stores, bar codes on sold products are optically 
scanned so as to acquire their identification numbers. 
The acquired identification numbers are used as a basis 
for retrieving previously stored prices of the correspond- 
ing products from a database, and the retrieved prices 
are displayed at the cashiers' counter. 
[0010] The conventional bar code has been utilized 
as a moans for alleviating the burden of cashiers enter- 
ing product identification numbers. However, the code 
system has not been used for bypassing the chores of 
input operations such as menu selections when a de- 
sired program is activated selectively from among ap- 
plication programs stored beforehand on a hard disk 
drive (HDD) of a general purpose personal computer. 
[0011] \terious respective aspects and features of the 
invention are defined in the appended claims. 
[0012] Embodiments of the invention relate to an im- 
age processing apparatus, an image processing meth- 
od, and a storage medium for allowing an image 
processing program to be executed by a computer, the 
apparatus and the method permitting acquisition of 
identification information from a two-dimensional code 
and position information about the two-dimensional 
code, the acquired information being used as a basis for 
carrying out. various processes. 

[001 3] Embodiments of the present invention can pro- 
vide an image processing apparatus, an image process- 
ing method, and a storage medium for allowing an im- 
age processing program to be executed by a computer, 
wherein a target image when simply captured prompts 
automatic execution of a desired process corresponding 
4o the image. 

[0014] In carrying out the invention and according to 
one aspect thereof, there is provided an image process- 
ing apparatus including: image acquiring means for ac- 
quiring a target image; identification information recog- 
nizing means for recognizing identification information 
corresponding to a specific image pattern from the tar- 
get image acquired by the image acquiring means; and 
activating means for activating selectively from among 
a plurality of previously stored processes a specific 
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process corresponding to the identification information 
recogni7ed by the identification information recognizing 
means so as to start execution of the specific process. 
[0015] According to another aspect of the invention, 
there is provided an image processing method including s 
the steps of: acquiring a target image; recognizing iden- 
tification information corresponding to a specific image 
pattern from the target image acquired by the image ac- 
quiring means; and activating selectively from among a 
plurality of previously stored processes a specific proc- 
ess corresponding to the identification information rec- 
ognized in the identification information recognizing 
step so as to start execution of the specific process. 
[0016] According to a further aspect of the invention, 
there is provided a storage medium for storing an image 
processing program executable by a computer and in- 
cluding the steps of: acquiring a target image; recogniz- 
ing identification information corresponding to a specific 
image pattern from the target image acquired by the im- 
age acquiring means; and activating selectively from 
among a plurality of previously stored processes a spe- 
cific process corresponding to the identification informa- 
tion recognized in the identification information recog- 
nizing step so as to start execution of the specific proc- 
ess. 

[0017] Through the use of the inventive image 
processing apparatus, image processing method and 
storage medium outlined above, identification informa- 
tion is first recognized from an acquired target image. A 
specific process corresponding to the acquired identifi- 
cation information is then activated selectively from 
among a plurality of previously stored processes. 
[001 8] The invention will now be described by way of 
example with reference to the accompanying drawings, 
throughout which like parts are referred to by like refer- 
ences, and in which: 

Fig. 1 is a schematic view of a personal computer 
to which the invention is applied; 
Fig. 2 is an explanatory view depicting specifica- 
tions of a two-dimensional code; 
Fig. 3 is a perspective view of a portable personal 
computer embodying the invention, with its display 
part swung open away from its body; 
Fig. 4 is a plan view of the computer in Fig. 1 ; 
Fig. 5 is a left-hand side view of the computer in Fig. 
1 with its display part swung shut onto its body; 
Fig. 6 is a right-hand side view of the computer in 
Fig. 1 with its display part swung open 1 80 degrees 
relative to its body; 

Fig. 7 is a front view of the computer in Fig. 3; 
Fig. 8 is a bottom view of the computer in Fig. 4; 
Fig. 9 is a block diagram showing an internal struc- 
ture of the personal computer in Fig. 1 ; 
Fig. 10 is a schematic view indicating a structure of 
an HDD in Fig. 9; 

Fig. 11 is a flowchart of steps constituting a launcher 
process; 



Fig. 1 2 is a schematic view sketching a typical dis- 
play on an LCD in Fig. 3; 

Fig. 1 3 is an explanatory view describing coordinate 
data representing a two-dimensional code: 
Fig. 1 4 is a flowchart of steps constituting a process 
of a 2D code prop/am; 



Fig. 15 is a view illustrating a typical image stored 
in a VRAM in Fig. 9; 



Fig. 16 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 17 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 18 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3: 












Fig. 19 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 20 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 21 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 22 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3; 












Fig. 23 is a schematic 


view 


of 


another 


display 


on 


the LCD in Fig. 3: 












Fig. 24 is a schematic 


view 


of 


another 


display 


on 



the LCD in Fig. 3; 

Fig. 25 is a flowchart of steps constituting a two- 
dimensional code storing process; 
Fig. 26 is a schematic view of a typical GUI display 
provided by a management tool program; 
Fig. 27 is a schematic view of another GUI display 
provided by the management tool program; 
Fig. 28 is a schematic view of another display on 
the LCD in Fig. 3: 

Fig. 29 is a schematic view of a portable telephone 
to which a two-dimensional code is attached; 
Fig. 30 is a schematic view of another display on 
the LCD in Fig. 3; and 

Fig. 31 is a schematic view of another display on 
the LCD in Fig. 3. 

[0019] Fig. 1 is a schematic view of a personal com- 
puter 1 to which an embodiment ol the invention is ap- 
plied. In this example, an object 100 is a business card. 
A two-dimensional code 101 is printed directly onto the 
right-hand side of the object 100, or a label on which the 
two-dimensional code 101 is printed is attached to the 
righl-hand side of the object 100. The two-dimensional 
code 101 may be referred to as the 2D code hereunder 
where appropriate. As shown in Fig. 2, the two-dimen- 
sional code 1 01 -constitutes a block unit-based rectangle 
measuring 9.5 blocks by 7 blocks. The rectangle com- 
prises a cell part A and a logo part B in black separated 
by a block. The cell part A forms a square of 7 blocks 
per side comprising square cells arranged in a two-di- 
mensional pattern. The logo part B includes a large- 
sized rectangle measuring 1.5 blocks by 7 blocks. The 
logo part B also has a logo mark such as "CyberCode 
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(registered trademark) printed as blanks representing a 
code system name of the two-dimensional code 101 . 
[0020] Illustratively, the following description is avail- 
able at the time of submitting this application from the 
home page offered by this applicant at (URL): 
http://www.sony.co.jp/sd/ProductsPark/Consumer/ 
. PCOM/PCG-C1 CATZcybercode.html 
[0021] "What we call "CyberCode" is Sony's unique 
two-dimensional code system that offers about 16.77 
million different patterns (in 24 bits). Of these patterns, 
about one million patterns (in 20 bits) may be registered 
as desired for program start-up purposes. The remain- 
ing code patterns are reserved for future service expan- 
sion. "CyberCode" works as an index to what is repre- 
sented by the code in question, the index allowing rele- 
vant information to be retrieved from computer storage. 
When a user starts a program through a new interface 
feature "CyberCode Finder," the user finds that the cor- 
responding information leaps from the object having the 
"CyberCode" onto the compuler screen." 
[0022] The personal computer 1 is a notebook type 
computer comprising a CCD video camera 23 located 
in a display part 3. Illustratively, the personal computer 

1 recognizes a pattern of the 2D code 101 on the basis 
of image data about the object 100 and 2D code 101, 
the image data being acquired by the CCD video cam- 
era 23. The personal computer 1 performs a process in 
accordance with the pattern of the 2D code 101 thus 
obtained. 

[0023] Figs. 3 through 8 depict a typical portable per- 
sonal computerto which an embodiment of the invention 
is applied. The personal computer 1 is a mini-notebook 
type personal computer that primarily comprises a body 

2 and a display part 3 attached swingingly to the body. 
2. Fig. 3 is a perspective view of the computer with the 
display part 3 swung open away from the body 2. Fig. 4 
is a plan view of the computer in Fig. 3. Fig. 5 is a left- 
hand side view of the computer with the display part 3 
swung shut onto the body 2. Fig. 6 is a right-hand side 
view of the computer with the display part 3 swung open 
1 80 degrees relative to the body 2. Fig. 7 is a front view 
of the computer in Fig. 5. Fig. 8 is a bottom view of the 
computer in Fig. 6. 

[0024] The face of the body 2 comprises a keyboard 
4 and a stick type pointing device 5. The keyboard 4 is 
used to input characters, symbols, etc., and the stick 
type pointing device 5 is operated to move a mouse cur- 
sor. Also furnished on Ihe body face is a speaker 8 out- 
putting sound and a shutter button 10 operated to take 
a picture using the CCD video camera 23 mounted on 
the display part 3. 

[0025] A pawl 13 is provided at the upper end of the 
display part 3. As shown in Fig. 5, with the display part 

3 swung closed onto the body 2, the pawl 1 3 hooks onto 
a hole 6 in the body 2. At the front of the body 2 is a slide 
lever 7 furnished in a crosswise movable fashion. The 
slide lever 7 is used to lock and unlock the pawl 13 so 
that the pawl 13 is engaged with and disengaged from 



6 

the hole 6. With the pawl 1 3 unlocked, the display part 
3 may be swung open away from the body 2. Adjacent 
to the pawl 1 3 is a microphone 24 which, as depicted in 
Fig. 8, may pick up sound from both the front and the 

5 back side of the body 2. 

[0026] The front of the body 2 further comprises a pro- 
grammable power key (PPK) 9. An air outlet 11 is pro- 
vided on the right-hand side of the body 2, as shown in 
Fig. 6. At the lower end in front of the body 2 is an air 

10 inlet 1 4 as depicted in Fig. 7. To the right of the air outlet 
1 1 is a slot 1 2 that accommodates a PCMCIA (Personal 
Computer Memory Card International Association) card 
(called a PC card). 

[0027] An LCD (liquid crystal display) 21 for displaying 

15 images is provided on the front of the display part 3. At 
the upper end of the LCD 21 is an image pickup part 22 
mounted rotatably on the display part 3. More specifi- 
cally, the image pickup part 22 is rotatable to any posi- 
tion within a range of 180 degrees in the same direction 

20 as the LCD 21 and in the opposite direction thereof (i. 
e., toward the back). The image pickup part 22 is fur- 
nished with the CCD video camera 23. 
[0028] At the bwer end of the display part 3 on the 
body side is a group of lamps including a power lamp 

2S p|_, a battery lamp BL a message lamp ML and other 
LEDs. Reference numeral 40 in Fig. 5 denotes a power 
switch furnished on the left-hand side of the body 2, and 
reference numeral 25 in Fig. 7 represents an adjusting 
ring used to adjust the focus of the CCD video camera 

30 23. Reference numeral 26 in Fig. 8 stands for a cover 
that conceals an opening through which to install an ad- 
ditional memory into the body 2, and reference numeral 
41 denotes a hole through which to insert a pin to unlock 
the cover 26. 

35 [0029] Fig. 9 illustrates an internal structure of the per- 
sonal computer 1 . An internal bus 51 is connected to a 
CPU (central processing unit) 52, a PC card 53 inserted 
as needed, a RAM (random access memory) 54, and a 
graphic chip 81 . The internal bus is coupled to an exter- 

40 nal bus 55. The external bus 55, for its part, is connected 
to a hard disk drive (HDD) 56, an I/O (input/output) con- 
troller 57, a keyboard controller 58, a stick type pointing 
device controller 59, a sound chip 60, an LCD controller 
83, and a modem 50. 

45 [0030] The CPU52 is a controller that controls diverse 
computer functions. The PC card 53 is installed as need- 
ed when an optional function is to be added. 
[0031] Image data captured by the CCD videocamera 
23 are forwarded to a processing part 82 for processing. 

50 The image data processed by the processing part 82 
are input to the graphic chip 81. The graphic chip 81 
stores the input video data into an internal VRAM 81 A, 
and retrieves the data from the memory as needed for 
output to the LCD controller 83. Given the image data 

55 from the graphic chip 81 , the LCD controller 83 outputs 
the data to the LCD 21 for display Back lights 84 are 
provided to illuminate the LCD 21 from the back. 
[0032] When the personal computer 1 is booted up, 
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an electronic mail program (an application program) 
54A, an auto pilot program (another application pro- 
gram) 54B and the OS (operating program) 54C are 
transferred from the HDD 56 to the RAM 54 and retained 
therein. 

[0033] The electronic mail program 54A is a program 
that exchanges communication messages with an ex- 
ternal entity using a communication line such as a tele- 
phone line and by way of a network. A received mail 
acquisition function is specifically included in the elec- 
tronic mail program 54A. The received mail acquisition 
function checks a mail server 93 to see if a mail box 93A 
therein contains any mail addressed to this program (i. 
e., to the user). If any such mail is found in the mail box, 
the received mail acquisition function carries out a suit- 
able process to acquire that mail. 
[0034] The auto pilot program 54B is a program that 
starts up and carries out a plurality of predetermined 
processes (or programs) in a predetermined sequence. 
[0035] The OS (operating system) 54C controls basic 
computer functions. Typical operating systems are Win- 
dows 95, Windows 98 (registered trademarks), and the 
like. 

[0036] As shown in Fig. 1 0, the hard disk drive (HDD) 
56 connected to the external bus 55 contains the elec- 
tronic mail program 56A, auto pilot program 56B, OS 
(operating system) 56C, a two-dimensional code data- 
base (called the 2D code database hereunder) 56D, a 
finder application program (called the FA program here- 
under) 56E, a two-dimensional code associating appli- 
cation program (called the 2D code program hereunder) 
56F, and a management tool program 56G. 
[0037] The 2D code database 56D stores, besides ID 
numbers of two-dimensional codes (called code IDs 
hereunder), names of executable files (programs) hav- 
ing an extension "exe" or ".com" each, or of document 
files such as those with extensions ".txt," ".htm," ".jpg," 
etc., established in connection with two-dimensional 
codes. An executable file is established so that it is au- 
tomatically started when the corresponding two-dimen- 
sional code is recognized (that file is called a 2D code- 
related executable file hereunder) . A document file is 
established so that it is automatically opened by a relat- 
ed executable tile which in turn is automatically activat- 
ed when the corresponding two-dimensional code is 
recognized (that file is called a 2D code-related docu- 
ment file hereunder). In the description that follows, 2D 
code-relaled executable files and 2D code-related doc- 
ument files may be collectively called 2D code-related 
files if there is no specific need to distinguish between 
the two types of files. 

[0038] In addition, the 2D code database 56D stores 
information describing contents of 2D code-related files 
in connection with code IDs (the information is called the 
2D code memo information hereunder). Also contained 
in the database is information attached to two-dimen- 
sional codes such as titles of 2D code-related files. In 
practice, the 2D code database 56D is accessed by use 



of various program modules called a DLL {dynamic link 
library) stored beforehand on the HDD 56. 
[0039] The FA program 56E illustratively carries out a 
two-dimensional code extracting process whereby im- 
s age data representing a given two-dimensional code are 
extracted from the image data held in the VRAM 81 A of 
the graphic chip 81 , a launcher process (described later) 
causing a process corresponding to a 2D code-related 
file to be automatically performed, and a visual effect 
10 process (described later) facilitating the use of these 
processes by the user. The FA program 56E also has 
an API (application programming interface) that is used 
to exchange code IDs and other diverse data with a 2D 
code program 56 F. 
15 [0040] The 2D code program 56F is a 2D code-related 
executable tile that may exchange data with the FA pro- 
gram 56E through the API of the latter. 
[0041] The management tool program 56G is used to 
store data necessary for the execution of the FA pro- 
gram 56E and 2D code program 56F. The management 
tool program 56G has a suitable GUI (graphical user in- 
terface). 

[0042] During the booting process, the OS 56C, auto 
pilot program 56B and electronic mail program 56 A are 
transferred successively from the hard disk drive 56 to 
the RAM 54 and stored in the memory. 
[0043] In this example, the 2D code database 56D is 
furnished on the HDD 56 of the personal computer 1. 
Alternatively, the 2D code database may be provided in 
a WWW (World Wide Web) server connected to the In- 
ternet 92 so that the database may be shared by users 
on the Internet. 

[0044] Returning to Fig. 9, the I/O controller 57 has a 
microcontroller 61 equipped with an I/O interface 62. 
The microcontroller 61 is constituted by the I/O interface 
62, a CPU 63, a RAM 64 and a ROM 69 which are in- 
terconnected. The RAM 64 includes a key input status 
register 65, an LED (light-emitting diode) control register 
66, a set time register 67, and a register 68. The set time 
register 67 is used to start the operation of a start se- 
quence controller 76 when a time preset by the user (i. 
e., starting condition) is reached. The register 68 holds 
a correspondence between a preset combination of op- 
eration keys (starting condition) on the one hand and an 
application program to be started on the other hand. 
When the user inputs the preset combination of opera- 
tion keys, the corresponding application program (e.g., 
electronic mail program) is started. 
[0045] When the fingertip-operated programmable 
power key (PPK) 9 is pushed, the key input status reg- 
ister 65 gets and retains an operation key flag. The LED 
control register 66 is used to control the illumination of 
the message lamp ML indicating that boot-up status of 
an application program (e.g., electronic mail program) 
which is held in the register 68. A desired time of day 
may be set to the set time register 67. 
[0046] The microcontroller 61 is connected to a back- 
up battery 74. The battery 74 allows contents of the reg- 
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isters 65, 66 and 67 to be retained when power to the 
body 2 is turned off. 

[0047] The ROM 69 in the microcontroller 61 contains 
in advance a wake-up program 70 ? a key input monitor- 
ing program 71 and an LED control program 72. The 
ROM 69 is illustratively composed of an EE PROM (elec- 
trically erasable and programmable read only memory) . 
The EEPROM is also called a flash memory. The micro- 
controller 61 is connected to an RTC (real-time clock) 
75 that keeps the current time. 

[0048] The wake-up program 70 in the ROM 69 is a 
program that checks to see if a preset time in the set 
time register 67 is reached on the basis of time-of-day 
data from the RTC 75. When the preset time is reached, 
the wake-up program 70 starts up a predetermined 
process (or program) . The key input monitoring pro- 
gram 71 continuously monitors whether the PPK 9 is 
pushed by the user. The LED control program 72 con- 
trols the lighting of the message lamp ML. 
[0049] Furthermore, the ROM 69 contains a BIOS 
(basic input/output system) 73. The BIOS is a software 
program that controls exchanges of data (input and out- 
put) between the OS or application software on the one 
hand and peripheral devices (e.g., display part, key- 
board, hard disk drive) on the other hand. 
[0050] The keyboard controller 58 connected to the 
external bus 55 controls input from the keyboard 4. The 
stick type pointing device controller 59 controls input 
from the stick type pointing device 5. 
[0051] The sound chip 60 receives input from the mi- 
crophone 24, and supplies sound signals to a built-in 
speaker 8. 

[0052] The modem 50 permits connection to a com- 
munication network 92 such as the Internet and to the 
mail server 93 through a public telephone line 90 and 
an Internet service provider 91 . 

[0053] The power switch 40 is operated to turn on and 
off the power supply. A half-push switch 85 is activated 
when the shutter button 10 is half -pushed. A full-push 
switch 86 is turned on when the shutter button 1 0 is fully 
pushed. A reverse switch 87 is turned on when the im- 
age pickup part 22 is rotated by 180 degrees (i.e., when 
the CCD video camera 23 is rotated into a direction suit- 
able for picking up an image on the opposite side of the 
LCD 21). 

[0054] When the launcher process programmed in 
the FA program 56E is carried out, the CPU 52 performs 
steps constituting a flowchart in Fig. 11 . The steps will 
now be described with reference to Fig. 11 . 
[0055] In step S1 , the CPU 52 boots the FA program 
56E stored on the HDD 56. In step S2, the CPU 52 waits 
for the shutter button 1 0 to be operated by the user. That 
is, the CPU 52 waits until the object 100 and the 2D code 
101 attached thereto are picked up by the CCD video 
camera 23, the resulting image data processed by the 
processing part 82, and the processed image data ren- 
dered in the VRAM 81 A of the graphic chip 81 . 
[0056] Illustratively, the user orientates that side of the 



object 100 which bears the 2D code 101 toward theCCD 
video camera 23, and operates the shutter button 10. 
This causes the CCD video camera 23 to capture image 
data about the 2D code 101. the captured image data 
5 being rendered in the VRAM 81 A of the graphic chip 81 . 
The image data are then displayed by the graphic chip 
81 onto a finder screen 201 of the LCD 21 by means of 
the LCD controller 83 as shown in Fig. 12. 
[0057] When the image data have been rendered in 
10 the VRAM 81 A of the graphic chip 81 in step S2, step 
S3 is reached. In step S3, the CPU 52 extracts the image 
data about the 2D code 101 from the image data ren- 
dered in the VRAM 8 1 A and checks to see if the 2D code 
is recognizable. If the 2D code 101 is successfully ree- 
fs ognized, the CPU 52 goes to step S5. The recognition 
corroborates the presence of the two-dimensional code 
in the image data captured by the CCD video camera 23. 
[0058] If the CPU 52 fails to recognize the 2D code 
101 in step S3, step S4 is reached. In step S4, the CPU 
52 causes the LCD controller 83 to display on Ihe LCD 
21 a message indicating the failed attempt to recognize 
the code, thus prompting the user to make another at- 
tempt. Step S4 is followed by step S2. 
[0059] In step S5, the CPU 52 acquires, from a cell 
pattern of the 2D code 1 01 recognized in stop S3, a code 
ID and coordinates of displayed rectangular cells mak- 
ing up four corners of the two-dimensional code ((x0, 
yO, zO), (x1 , y1 , z1 ), (x2, y2, z2); called 2D code coordi- 
nate data hereunder) shown in Fig. 1 3. In the description 
that follows, the code ID and the 2D code coordinate 
data may be collectively called 2D code recognition in- 
formation if there is no specific need to distinguish be- 
tween the two. 

[0060] In step S6, the CPU 52 checks to see if certain 
fields of the FA program 56E store the code ID acquired 
in step S5 and a window handle (window ID) of the cor- 
responding 2D code program 56F If no such ID infor- 
mation is found, step S7 is reached. In the description 
that follows, the code ID held in a certain field of the FA 
program 56E and the window handle of the correspond- 
ing 2D code program 56F may be collectively called 2D 
code program storage information. 
[0061] In step S7, the CPU 52 checks the 2D code 
database 56D to see if the database contains the code 
ID acquired in step S5. If the code ID is found to be 
stored, step S8 is reached. In step S8, the CPU 52 
searches the 2D code database 56D for the name of a 
2D code-related file stored in correspondence with the 
code ID in question. A check is then made to see which 
file is setcorresponding to the 2D code 1 01 in a 2D code- 
related executable file, or a 2D code-related document 
file. 

[0062] In stepS8, if the CPU 52 judges that a 2D code* 
related executable file is set with respect to the 2D code 
101 , step S9 is reached. In step S9, a check is made to 
see if the 2D code-related executable file is the 2D code 
program 56 F 

[0063] In step S9. if the CPU 52 judges the 2D code- 
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related executable file to be the 2D code program 56F, 
step S10 is reached. In step S10, the 2D code program 
56F is started. 

[0064] In step S11. the CPU 52 stores the window 
handle of the started 2D code program 56F into a suit- 
able field of the FA program 56E in correspondence with 
the code ID acquired in step S5. This allows the 2D code 
program 56F to be written to the FA program 56E. 
[0065] In step S1 2, the CPU 52 outputs a message to 
the window handle stored in the FA program 56E. The 
message in this case is a 2D code recognition message 
indicating that the two-dimensional code has been rec- 
ognized, i.e., that 2D code recognition information about 
the 2D code 101 has been acquired. In this manner, the 
2D code program 56F is notified of the 2D code recog- 
nition message. 

[0066] In step S1 3, the CPU 52 supplies the 2D code 
program 56F, through the API, with the 2D code recog- 
nition information about the 2D code 101 acquired in 
slep S5. In turn, the 2D code program 56F performs 
processing based on the 2D code recognition informa- 
tion (e.g., 2D code coordinate data about the 2D code 
101). 

[0067] With the processing of the 2D code program 
56F cndcd : stop S14 is reached. In step S14, the CPU 
52 causes the LCD controller 83 to display a message 
prompting the user to delete the 2D code program stor- 
age information from a predetermined field of the FA 
program 56E. In response to the message, the user de- 
letes the 2D code program storage information. 
[0068] In step S1 5, the CPU 52 checks to see if a com- 
mand is received from the keyboard controller 58 or stick 
type pointing device controller 59 designating erasure 
of the 2D code program storage information. If that com- 
mand is found to be received, step S16 is reached. In 
step S16, the 2D code program storage information is 
deleted. Step S1 6 is followed by step S2. If the CPU 65 
receives in step S15 a command indicating the user's 
willingness to forgo erasure of the 2D code program 
storage information, then step S16 is skipped and step 
S2 is reached. 

[0069] If the 2D code program 56F is executed con- 
secutively, the 2D code program storage information 
may be left stored in a predetermined field of the FA pro- 
gram 56E. This permits more efficient execution of the 
processing. Illustratively, suppose that in step S6, the 
2D code program storage information is judged to exist 
in a certain field of the FA program 56E. In that case, 
steps S7 through S11 are skipped and step S12 is 
reached. Thereafter, the newly acquired 2D code rec- 
ognition information is fed to the 2D code program 56 F 
which in turn carries out an appropriate process. 
[0070] If the CPU 52 judges in step S7 that the code 
ID acquired in step S5 is not contained in the 2D code 
database 56D, then step S17 is reached. In step S17, 
the CPU 52 starts up the management tool program 56G 
(described later). Step S17 is followed by step S2. 
[0071] If the CPU 52 judges in step S8 that a 2D code- 



12 

related document file is designated with regard to the 
code ID, step S18 is reached. In step S18, the 2D code- 
related document file is opened. Step S18 is followed 
by step S2. 

5 [0072] If the CPU 52 judges in step S9 that an exe- 
cutable file other than the 2D code program 56 F is des- 
ignated as a 2D code-related executable file, step S19 
is reached. In step S19, the executable file in question 
is started. Step S1 9 is followed by step S2. In this case, 

10 although the executable program is started up, unlike in 
the case of the 2D code program 56F, nothing is report- 
ed thereafter by the FA program 56F despite inputs 
made by the user. 

[0073] When the 2D code 101 is recognized in the 
is manner described above, the 2D code-related file es- 
tablished in connection with that code is automatically 
executed. 

[0074] How the CPU 52 works in conjunction with the 
steps of the 2D code program 56F will now be described 
wilh reference to a flowchart of Fig. 14. 
[0075] When the 2D code program 56F is started in 
stepSIOof Fig. 11, the CPU 52 enters step S21. Instep 
S21 , the CPU 52 supplies the FA program 56E with that 
window handle of the 2D code program 56 F which is 
held in the FA program 56E so that the window handle 
is stored in the FA program 56E (this step corresponds 
to step S11). 

[0076] In step S22, the CPU 52 receives a 2D code 
recognition message from the FA program 56E (corre- 
sponding to step S1 2). In step S23, the CPU 52 acquires 
2D code recognition information from the FA program 
56E through the API (corresponding to step S13). 
[0077] In step S24, the CPU 52 carries out a suitable 
process based on the 2D code recognition information 
acquired in step S23. This example involves having a 
face image 1 1 0 in Fig. 1 5 superimposed onto, say, a blue 
rectangular background portion 1 20 (shaded) printed on 
the left hand side of the object 100 shown in Fig. 12. It 
is assumed here that motion image files such as an an- 
imation GIF file for displaying the face image 11 0 in Fig. 
15 are stored beforehand on the HDD 56 or somewhere 
appropriate. 

[0078] The CPU 52 thereupon calculates that position 
on the LCD 21 in which to display the face image 110 
on the basis of the 2D code coordinate data about the 
2D code 101 , the data being derived from the 2D code 
recognition information on the 2D code 101 acquired in 
step S23. For example, the coordinates denoting four 
corners of the background portion 1 20 of the object 100 
are obtained in correspondence with the 2D code coor- 
dinate data about the 2D code 1 01 . 
[0079] The CPU 52 reads from the HDD 56 the image 
file for displaying the face imago 110 and, based on a 
calculated display position of the face image 110, for- 
mulates a composite image by rendering in the VRAM 
81 A the face image 1 10 together with images of the ob- 
ject 100 and 2D code 101. With the composition of the 
images finished, the graphic chip 81 causes the LCD 
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controller 83 to display the image data rendered in the 
VRAM 81 A onto the finder screen 201 of the LCD 21, 
as shown in Fig. 16. 

[0080] It is possible to get the face image 110 dis- 
played on the finder screen 201 of the LCD 21 in any 
one of several manners: as an animated picture based 
on animation GIF files or the like, as a high-quality mo- 
tion image using MPEG files, or as a still image employ- 
ing JPEG files. A motion image may be displayed in syn- 
chronism with a sound reproduced from a voice file cor- 
responding to the image. 

[0081] When the 2D code coordinate data acquired 
as 2D code recognition information, i.e., position infor- 
mation about the 2D code 101, is fed to the 2D code 
program 56F as described, the image composing proc- 
ess, among others, is made possible. That operating 
principle applies to another example: when, as shown 
in Fig. 17, four menu selection buttons A through D are 
displayed on the finder screen 201 of the LCD 21 , a two- 
dimensional code may be brought into display in a po- 
sition where a desired menu selection button is shown. 
This causes the desired menu selection button to be se- 
lected. In that manner, the user can select any menu 
selection button without operating the keyboard 4 or 
stick typo pointing device 5. 

[0082] Described below with reference to Figs. 1 8 and 
1 9 are steps constituting a visual effect process. The 
process, programmed in the FA program 56E, is carried 
out during execution of the above-described 2D code 
program 56F (2D code-related executable file). 
[0083] When the image of the 2D code 101 such as 
one shown in Fig. 12 is displayed on the finder screen 
201 of the LCD 21 . the CPU 52 detects 2D code recog- 
nition information about the recognized 2D code 101 
(corresponding to step S5). On detecting the informa- 
tion, the CPU 52 causes the LCD controller 83 to put a 
frame around the displayed 2D code 101 on the finder 
screen 201. This allows the user to confirm with ease 
that the 2D code 101 has been recognized. 
[0084] The CPU 52 searches the 2D code database 
56D for a 2D code-related file name stored in corre- 
spondence with the code ID of the 2D code 101. When 
the 2D code-related file name is retrieved, the CPU 52 
acquires a handle of a pictorial icon 121 designed to rep- 
resent the content of the 2D code-related file. When the 
handle is acquired, the CPU 52 causes the LCD control- 
ler 83 to superimpose it onto the 2D code 1 01 as shown 
in Fig. 19. The CPU 52 further searches the 2D code 
database 56D for 2D code memo information stored in 
connection with the code ID. When the 2D code memo 
information is retrieved, the CPU 52 causes the LCD 
controller 83 to indicate the information in a suitable dis- 
play portion 202 on the LCD 21 as shown in Fig. 1 9. The 
indication allows the user to know easily the content of 
the process performed by use of the 2D code-related 
file associated with the 2D code 101 . 
[0085] The image of the icon 121 appearing super- 
posed on the 2D code 101 may illustratively be rotated 



like a miniature globe. Such a highlighted display pro- 
vides easy confirmation of the current status by visually 
informing the user that the 2D code-related file has been 
successfully retrieved. 
5 [0086] While the process corresponding to the 2D 
code-related file is being carried out, the CPU 52 causes 
the LCD controller 83 to expand gradually in four direc- 
tions the frame 130 surrounding the 2D code 101, as 
indicated by broken-line frames 1 31 , 1 32 and 1 33 in Fig. 
1 9. The expanding display allows the user to verify eas- 
ily that the process associated with the 2D code-related 
file is being executed. 

[0087] Described below with reference to Figs. 20 
through 24 are other steps also constituting a visual ef- 
is feet process. This process, programmed in the FA pro- 
gram 56E, is carried out during execution of a process 
associated with a 2D code-related document file. 
[0088] Suppose that an image of an MD (Mini Disc; 
registered trademark) 150 and a 2D code 151 repre- 

20 senling an "ALBUM" file containing information about 
pieces of music recorded on the MD 150 are captured 
by the CCD video camera 23 and displayed on the finder 
screen 201 of the LCD 21 as shown in Fig. 20. In that 
case, the CPU 52 acquires 2D code recognition infor- 
ms mation about the recognized 2D code 151 (correspond- 
ing to step S5). The CPU 52 thereupon causes the LCD 
controller 83 to display a cross mark in the middle of the 
finder screen 201 and an indication "SEARCH IN 
PROGRESS" at the bottom of the finder screen 201 as 

30 depicted in Fig. 21 . This allows the user to ascertain that 
the 2D code 151 has been recognized. 
[0089] The CPU 52 searches the 2D code database 
56D for 2D code memo information stored in corre- 
spondence with the code ID of the 2D code 151. When 

35 the 2D code memo information is retrieved, the CPU 52 
causes the LCD controller 83 to display an indication 
"ACQUIRED" at the bottom of the finder screen 201 as 
shown in Fig. 22 along with the 2D code memo informa- 
tion in the display portion 202. In this example, the name 

40 of a singer D is shown in the display portion 202 as the 
2D code memo information. The display allows the user 
to verify easily that the MD 150 contains songs by the 
singer D. Alternatively, a face image of the singer D may 
be displayed as the 2D code memo information. 

45 [0090] When starting to open the "ALBUM" file, i.e., a 
2D code-related file set for the 2D -code 105, the CPU 
52 causes the LCD controller 83 to give an indication 
"OPENING" at the bottom of the finder screen 201 to- 
gether with a time count indicating the time it takes to 

so complete the opening process. With the "ALBUM" file 
thus opened, song titles, playing times and other infor- 
mation held in the "ALBUM" file are displayed as illus- 
trated in Fig. 24. 

[0091] How the CPU 52 works in conjunction with 
55 steps of the management tool program 56G will now be 
described with reference to a flowchart of Fig. 25. In this 
example, necessary data or commands are transmitted 
to the CPU 52 through a GUI corresponding to the man- 
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agement tool program 56G such as is shown in Figs. 26 
and 27. 

[0092] The management tool program 56G is auto- 
matically started (in step S17) if the code ID of the rec- 
ognized two-dimensional code is not found in the 2D 
code database 56D (step S7 in Fig. 11). Alternatively, 
the management tool program 56G is started by the user 
operating the keyboard 4 or stick type pointing device 
5. In any case, the management tool program 56 is start- 
ed in step S31 . Step S31 is followed by step S32 in which 
a check is made to see if a code ID of the two-dimen- 
sional code to be stored is designated. If no such code 
ID is found designated, step S33 is reached. 
[0093] In step S33, the CPU 52 causes the LCD con- 
troller 83 to display a code ID setting GUI 300 (Fig. 26) 
on the LCD 21 . In step S34, the CPU 52 waits for a code 
ID input part 301 to receive a code ID of the two-dimen- 
sional code to be stored. The user enters the code ID 
either directly into the code ID input part 301 by operat- 
ing the keyboard 4 or slick type poinling device 5, or 
indirectly by operating code ID feed buttons 303. When 
the code I D of the target two-dimensional code has been 
input by one of the above methods to the code ID input 
part 301, step S35 is reached. In step S35, the CPU 52 
stores the input code ID into the 2D code database 56D 
on the HDD 56. 

[0094] If, in step S32, the CPU 52 judges that the code 
ID of the two-dimensional code to be stored is designat- 
ed, then step S35 is reached. Illustratively, when 2D 
code recognition information about the two-dimensional 
code is acquired in step S5 of Fig. 11 . a cell pattern of 
the two-dimensional code is displayed in the 2D code 
display part 302, and the code ID is indicated in the code 
ID input part 301. In such a case, the CPU 52 judges 
that the code ID is designated and, goes to step S35. 
[0095] In step S36, the CPU 52 causes the LCD con- 
troller 83 to display a 2D code-related file setting GUI 
400 shown in Fig. 26. In step S37, the CPU 52 waits for 
an assigned file name input part 304 to receive a 2D 
code-related file name to be set in conjunction with the 
two-dimensional code to be stored. The user either se- 
lects a 2D code-related file from among the names of 
recently used 2D code-related files displayed in a dis- 
play field 305, or chooses by operating a reference but- 
ton 306 a 2D code-related file from other than the files 
indicated in the display field 305. When the 2D code- 
related file is selected by one of the above methods and 
the file name is input to the assigned file name input part 
304, the CPU 52 goes to step S38. 
[0096] In step S38, the CPU 52 waits for a 2D code 
setting button 307 to be operated on the 2D code-related 
file setting GUI 400. When the button 307 is operated, 
step S39 is reached. In stop S39, the CPU 52 causes 
the LCD controller 83 to display a set data creating GUI 
500 on the LCD 21 as shown in Fig. 27. 
[0097] In step S40, the CPU 52 waits until the title of 
a 2D code-related file is input to a title input part 401 of 
the set data creating GUI 500, until 2D code memo in- 



formation is input to a 2D code memo information input 
part 402, and until a verification button 403 is operated. 
When the title and the 2D code memo information are 
input and the verification button 403 is operated, step 

s S41 is reached. In step S41 , the CPU 52 stores into the 
2D code database 56D the name of the 2D code-related 
file selected in step S37 as well as the information en- 
tered in step S40. The file name and the information are 
stored in correspondence with the code ID. 

10 [0098] An importance affirmation button 404 on the 
set data creating GUI 500 is operated in order to prevent 
automatic erasure of a stored two-dimensional code 
101. Where the storage capacity of the 2D code data- 
base 56D is exhausted, efficient data management is 

is implemented illustratively by automatically erasing the 
least-recently accessed two-dimensional codes 101 or 
the two-dimensional codes 101 retained past a prede- 
termined effective date. In such cases, the importance 
affirmation button 404 is operated to designate a two- 

20 dimensional code 101 which, because of its importance, 
should not be automatically erased for data manage- 
ment purposes at a later date. 

[0099] In step S42, the CPU 52 checks to see if a de- 
tailed data setting command is received from the key- 

25 board controller 58 or stick type pointing device control- 
ler 59. If the command is found to be received, step S43 
is reached. In step S43, the CPU 52 causes the LCD 
controller 83 to display an appropriate GUI on the LCD 
21 and to store information input through the GUI into 

30 the 2D code database 56D in correspondence with the 
code ID. The process is then terminated. If the detailed 
data setting command is not found to be input in step 
S42. the CPU 52 skips step S43 and ends the process- 
ing. 

35 [0100] In the manner described, the 2D code-related 
file is established in conjunction with the two-dimension- 
al code. 

[0101] The storing process by the management tool 
program 56G will now be described in more specific 

40 terms. Illustratively, suppose that a file named "TO TEL- 
EPHONE" describing a matter that the user wants to 
communicate to a friend H is created and retained as 
shown in Fig. 28. The "TO TELEPHONE" file is then 
stored as a 2D code-related document tile in connection 

45 with a 2D code 501 attached to a portable telephone 
500 depicted in Fig. 29. 

[0102] First, the CCD video camera 23 picks up an 
image of the 2D code 501 . The CPU 52 recognizes the 
2D code 501 (step S3) and acquires a code ID corre- 

50 sponding to the 2D code 501 (step S5). Then the CPU 
52 judges that the acquired code ID is not stored in the 
2D code database 56D (step S7) t and starts the man- 
agement tool program 56G accordingly (step S17). 
[0103] The CPU 52 causes the LCD controller 83 to 

55 display on the LCD 21 both the 2D code-related file set- 
ting GUI 400 of Fig. 26 and the set data creating GUI 
500 of Fig 27. The user selects the previously stored 
"TO TELEPHONE" file by operating the keyboard 4 or 
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stick type pointing device 5, and sets the file in conjunc- 
tion with the 2D code 501 . 

[0104] With the above settings completed, next time 
the user gets the CCD video camera 23 to capture the 
2D code 501 on the portable telephone 500, the launch- 
er process is carried out. As shown in Fig. 30, the "TO 
TELEPHONE" file is then opened automatically and its 
content displayed on the LCD 21 . Checking the display, 
the user is reminded that the matter should be commu- 
nicated to the friend H. 

[0105] In the examples above, the invention was 
shown applied to two-dimensional codes. Alternatively, 
the invention may be applied to getting a user's V sign 
image 601 captured by the CCD video camera 23 and 
subjected to such image processing as edge extraction 
as shown in Fig. 31. When the user's predetermined 
sign or gesture is recognized as an image pattern like 
the one in Fig. 31 , a specific program corresponding to 
that sign or gesture may be arranged to be executed. 
[0106] A computer program designed to perform the 
above -described processes may be retained on such 
package media as floppy disks, CD-ROMs and DVDs; 
on semiconductor memories, magnetic disks and the 
like where the program is stored temporarily or perma- 
nently; on wired and wireless communication media 
such as local area networks, the Internet, and digital sat- 
ellite broadcasting networks: or in diverse communica- 
tion interfaces such as routers and modems for trans- 
mitting or receiving the program offered by the foregoing 
media. Such media, networks, interfaces and other 
measures allow the program to be installed in comput- 
ers for program execution. The storage medium as men- 
tioned in this specification refers broadly to all such me- 
dia, networks, interfaces and measures. 
[0107] As described and according to the inventive 
image processing apparatus, image processing method 
and storage medium above, identification information is 
first recognized from an acquired target image. A spe- 
cific process corresponding to the acquired identifica- 
tion information is then activated selectively from among 
a plurality of previously stored processes. Thus, merely 
providing a target image automatically triggers execu- 
tion of the process corresponding to that image. 
[0108] As many apparently different embodiments of 
this invention may be made without departing from the 
[0109] scope thereof, it is to be understood that the 
invention is not limited to the specific embodiments 
thereof except as defined in the appended claims. 
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sponding to a specific image pattern from said 
target image acquired by said image acquiring 
means; and 

activating means for activating selectively from 
among a plurality of previously stored process- 
es a specific process corresponding to said 
identification information recognized by said 
identification information recognizing means so 
as to start execution of said specific process. 

2. An image processing apparatus according to claim 
1, wherein said identification information recogniz- 
ing means extracts a visible code attached to said 
target from said target image acquired by said im- 
age acquiring means and recognizes identification 
information corresponding to an image pattern of 
said visible code. 

3. An image processing apparatus according to claim 
1 , wherein said identification information recogniz- 
ing means extracts a two-dimensional code at- 
tached to said target from said target image ac- 
quired by said image acquiring means and recog- 
nizes identification information corresponding to an 
image pattern of said two-dimensional code. 

4. An image processing apparatus according to claim 
1, further comprising: 

target image displaying means for displaying 
said target image acquired by said image ac- 
quiring means inside a specific display area; 
position information detecting means for de- 
tecting position information about said specific 
image pattern relative to said target image ac- 
quired by said image acquiring means; and 
rendering means for rendering an image corre- 
sponding to said identification information rec- 
ognized by said identification information rec- 
ognizing means, inside said specific display ar- 
ea and in a position corresponding to said po- 
sition information detected by said position in- 
formation detecting means. 



Claims 



45 5. An image processing apparatus according to claim 
4, wherein said rendering means renders said im- 
age corresponding to said identification information 
recognized by said identification information recog- 
nizing means in a superposed manner onto said tar- 

so get image displayed by said target image displaying 
means inside said specific display area. 



1. An image processing apparatus comprising: 

image acquiring means for acquiring a target ss 
image; 

identification information recognizing means 
for recognizing identification information corre- 



An image processing apparatus according to claim 
4, wherein said rendering means renders an icon 
representing contents of said specific process 
which is activated by said activating means and 
which corresponds to said identification informa- 
tion, in a superposed manner ontosaid target image 
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displayed inside said specific display area. 

7. An image processing apparatus according to claim 
A, wherein said rendering means renders an anima- 
tion representing contents of said specific process 
which is activated by said activating means and 
which corresponds to said identification informa- 
tion, in a superposed manner onto said target image 
displayed inside said specific display area. 

8. An image processing apparatus according to claim 
4, wherein said rendering means renders an image 
obtained as a result of said specific process which 
is activated by said activating means and which cor- 
responds to said identification information, inside 
said specific display area. 

9. An image processing apparatus according to claim 
4, wherein said rendering means renders a motion 
image obtained as a resull of said specific process 
which is activated by said activating means and 
which corresponds to said identification informa- 
tion, inside said specific display area. 

10. An image processing apparatus according to claim 
1, further comprising execution content displaying 
means for displaying character information associ- 
ated with said specific process which is activated 
by said activating means and which corresponds to 
said identification information. 

11. An image processing apparatus according to claim 
1 , further comprising storing means for storing be- 
forehand and in a desired manner said specific 
process which corresponds to said identification in- 
formation recognized by said identification informa- 
tion recognizing means and which is activated by 
said activating means. 

12. An image processing method comprising the steps 
of: 

acquiring a target image; 
recognizing identification information corre- 
sponding to a specific image pattern from the 
target image acquired by said image acquiring 
means; and 

activating selectively from among a plurality of 
previously stored processes a specific process 
corresponding to said identification information 
recognized in said identification information 
recognizing step so as to start execution of said 
specific process. 

13. A storage medium for storing an image processing 
program executable by a computer and comprising 
the steps of: 



acquiring a target image; 
recognizing identification information corre- 
sponding to a specific image pattern from the 
target image acquired by said image acquiring 

s means; and 

activating selectively from among a plurality of 
previously stored processes a specific process 
corresponding to said identification information 
recognized in said identification information 

w recognizing step so as to start execution of said 

specific process. 

14. A computer program having program code for per- 
forming a method according to claim 12. 
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